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ABSTRACT 


Aerospace vehicles are produced in limited quantities that do not always allow development of MIL- 
HDBK-5 A-basis design allowables. 1 One method of examining production and composition variations is to 
perform 100% lot acceptance testing for aerospace Aluminum (Al) alloys. This paper discusses statistical trends 
seen in strength data for one Al alloy. A four-step approach reduced the data to residuals, visualized residuals as a 
function of time, grouped data with quantified scatter, and conducted analysis of variance (ANOVA). 


BACKGROUND 

A test article was fabricated to demonstrate full-scale performance of a production-sized alloy ingot, as 
well as to qualify rolling, annealing, and weld approaches. Each panel was subjected to 100% lot acceptance testing 
to verify specification compliance. The test directions were: 

• Longitudinal (L) or parallel to the rolling direction 

• Long transverse (LT) or perpendicular to the rolling direction in the plane of the plate 

• Short transverse (ST) or perpendicular to the rolling direction in the plane of the plate 

• 45° from the rolling direction in the plane of the plate 

This lot acceptance testing involved multiple tiers of testing and evaluation. Each lot was sampled by 
machining test coupons in the L, LT, and 45° directions. The coupons were then subjected to tensile tests to 
determine ultimate tensile strength (UTS), yield strength (YS), and elongation. Thicker plates were also tested in 
the ST direction. Failed lots were retested twice to confirm the validity of the original test. If possible, failed lots 
were recovered by solution heat treatment. Failure stresses in this plate material followed a severely truncated 
distribution. Plates with nominal values were easily found, but plates with marginal values were rare. 

The first step in quality control is to verify whether the current distribution is acceptable. If not, the process 
used to fabricate the parts is improved to produce material with a tighter distribution of critical properties. Here, the 
distribution appears acceptable and sufficient sampling has been performed to verify the process produces few plates 
with unacceptable properties. The second step is to introduce continual monitoring to ensure the parts remain 
defined by the acceptable distribution. This step is more difficult to implement because the distribution has a sharp 
drop-off after lot acceptance testing. 


Process control techniques normally require the presence of some distance between the upper control limit 
(UCL) and the upper specification limit (USL). The UCL is often the mean plus three sigma, which is a probability 
<1/740 of having an out-of-bounds signal generated by an in-control process. If the process begins to drift, samples 
begin to produce values above the UCL. This development is a signal that the process must be brought under 
control again before an out-of-specification part is produced. However, the UCL is close to the mean in this alloy, 
due to the sharp drop-off of the distribution. If the USL is placed near the UCL, only a small probability exists that 
the USL would be exceeded if the current distribution continued. As a result, process malfunctions would probably 
not be caught before unacceptable parts had been produced. 

The alloy specifications included requirements for both strength and toughness. These properties cannot be 
independently controlled. Process operations intended to increase toughness inadvertently decrease strength, while 
increases in strength are generally accompanied by decreases in toughness. What might appear to be two single- 
sided distributions of bulk strength and toughness is actually a single double-sided distribution of the plate’s 
thermomechanical history. The anisotropy of most rolled A1 plate results in different properties for different 
directions. Each plate thickness contains a range of microstructures distributed throughout the plate. Intentional 
process variations include different amounts of cold work and heat treatment used for various parts of the tank. 
Gradual process changes may also lead to property or variability changes. 

A statistical analysis was performed on lot acceptance data for an A1 alloy intended for use in an aerospace 
vehicle. A-basis design allowables were calculated based on MIL-HDBK-5 procedures. Some property 
distributions were Normal, others Weibull. A lack of process-stable data limits the usefulness of this analysis. 

This paper discusses data taken early in an aerospace project to provide approaches to analyzing lot 
acceptance data for gradual changes in properties. The data constitute 400 lots and 3,200 specimens. 


ANALYSIS 

In Figure 1, UTS and YS are plotted for all samples to demonstrate the correlation of strength with 
processing. OM-temper data fall in the lower left-hand comer, T-3 temper data in the middle, and T-8 temper data 
in the upper right-hand comer. X-Y plots usually show the dependence of one variable (y) on another (x), with YS 
or UTS plotted as a function of intentional process variations. However, these processes are difficult to express as a 
single dimension. Instead of showing cause (x) and effect (y), this plot represents two effects (YS and UTS) of the 
same cause (processing). 



Figure 1 . UTS versus YS 



Figure 1 demonstrates the effect of intentional differences in processing. As expected, materials with high 
YS also have high UTS. Deviations from a simple thin line are due to random rather than systematic variation. A 
direct comparison would normalize values for systematic changes in process and geometry by plotting differences 
between UTS and YS and averages for each sample type. Such comparisons might be set up for different values of 
heat treatment, thickness, orientation, etc. The residual is the difference between the expected value based upon all 
input variables and the actual value for a given point. 2 Intentional variations are extracted to leave the effect of 
unintentional process variations intact. 

Figures 2 through 6 show results of the four-step method used for this study. Figure 2 shows analysis 
results for the first step (reducing the data to residuals) and compares residuals for UTS and YS. This plot would 
look different had all the data fallen at either extreme. One extreme distribution might have been plotted as a thin 
straight line, with all variation due to unintentional process variation. Another might have appeared as a circle 
around the origin, with all variation due to such random effects as inherent material variability or measurement 
error. The correlation coefficient is used to quantify the degree of variation and randomness. Here, the correlation 
coefficient squared of the data group is 80%. This result indicates that 80% of the property variation is due to 
variations in process and 20% is caused by random effects. 





UTS deviations 


Figure 2. UTS versus YS residuals 




Figure 3 shows analysis results for the second step (visualizing residuals as a function of time) with UTS 
residuals plotted by test date. Time is used as a proxy for process improvements based on the assumption that the 
processes continuously changed. Such process improvements might include a new heat treat control philosophy, 
composition retargeting, rolling practices, etc. The data do not show any obvious drift toward progressively higher 
UTS or a narrowing of the data envelope, which would indicate better process control. Residuals are considered a 
good metric for process control, with large values indicating significant process variations. 
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Figure 3. UTS residuals versus time 




Figure 4 shows analysis results for the second and third steps (grouping data and quantifying scatter). 
Twelve data groups may develop when residuals are divided into subgroups to facilitate measurements of data 
scatter and mean. These boundaries reflect natural divisions between testing efforts and keep the size of each group 
within one order of magnitude of the other groups. Unlike the UTS data, the elongation data show a gradual 
narrowing of the data envelope. 
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Figure 4. Elongation residuals versus time 

Figures 5 through 6 show other analysis results for the third step. Here, Figure 5 shows the size of the 
groups generated. All data groups fall between 60 and 600 points, except for the two groups at the ends. 



Figure 5. Number of points versus time 




Figure 6 includes lines that represent averages for each group. These data do not indicate a trend with time. 
The error bars indicate plus and minus one variance (i.e.: one standard deviation squared) while short horizontal 
lines show maxima and minima for data groups. Neither variance nor range shows any systematic narrowing with 
time, although they show significant differences from group to group. High variance occurs together for YS and 
UTS, primarily due to process variations. 
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Figure 6. Summary of Variance for Three Properties 
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Figure 6. Summary of Variance for Three Properties (continued) 


Process variations are considered the primary source of property variability. No clear patterns emerge 
when raw residuals are examined as a function of time or when discrete groups of data are analyzed separately. 
Data variation is quantified as a function of input variables when ANOVA 3 is used to track process control for sets 
of acceptance data. However, this technique presents an inherent problem. A well-designed ANOVA experiment 
should have the same number of data points for each combination of input variables. If the data points are not 
evenly distributed, the input variables are considered to be convoluted. For example, Table 2 shows convolution 
when a number of data points are compared as a function of heat treatment and time. 


Table 2 - Number of Samples Tested 
(Heat Treatment versus Time) 


Start Date 
(1 st test run) 
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- 

- 

- 

- 

- 

15 


Note: 


Two tempers were applied to the material, the first after it was roiled and the second after it was slightly stretched. 
“OM" is a temper designation which means that the material was not heat treated during this step. 




























Several blanks appear in the matrix. However, many spaces can be eliminated by culling the data. Table 3 
shows a new matrix that should produce better ANOVA results, although the refined data are still convoluted. 


Table 3 - Refined Data 
(Heat Treatments versus Time) 


Start Date 
(l" test run) 

Tempers 1 

(DOM 

(1) OM 

(DT3M4 

(1)T3M4 

(1) T8M4 

(2) OM 

(2) T8L4 

(2) T3M4 

(2) T8L4 

(2) T8M4 

Year 1 
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Year 3 

2 

12 

5 

40 

98 
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59 

5 
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5 

24 

1 

8 
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Note: Two tempers were applied to the material, the first after it was rolled and the second after it was slightly stretched. 

“OM 1 ' is a temper designation which means that the material was not heat treated during this step. 


Table 4 shows ANOVA results for the refined data, which involve four different gage thicknesses. Two 
thin gages were heat treated under one set of conditions while two thick gages were heat treated under another set of 
conditions. Such differences indicate that the data will be convoluted for plate thickness and heat treat variables. 
This situation was addressed by combining the thinnest gages into one group for a given set of heat treatments. 


Table 4 - Sources of Variance 
Indicated by ANOVA Results for Culled Data 


Variable 

Number of Levels 

UTS (%) 

YS (%) 

Elongation (%) 

Time 

10 

3 

3 

5 


2 

2 

2 

2 

Direction 

4 

17 

19 

32 

Heat Treatment 

5 

78 

76 

61 

Error 

- 

0 

0 

0 


These results are not as precise as they would be for well-conditioned data resulting from a designed 
experiment. However, the ANOVA suggests that anisotropy and heat treatment control over 90% of the data 
variability studied. Small but significant contributions also come from gage thickness and time. Together, these 
four variables account for most of the variability seen in this particular A1 alloy. 


CONCLUSIONS 

1 . This study considered a large set of lot acceptance data for an aerospace A1 alloy. However, the final ANOVA 
results are not entirely accurate because the original data inputs were convoluted. 

2. Over 90% of the data variability can be attributed to the effects of anisotropy and heat treatment. 

3. Process variation did not systematically increase or decrease during this period of time. 
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